FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
nvisentin_FTNT
Article Id 190055

Description

 

This article describes how to use BGP to advertise routes and SD-WAN for path selection.

Scope

 

From FortiOS 6.0, the SD-WAN feature supports dynamic routing.

 

This article references SD-WAN configuration as it appears in FortiOS v6.
In FortiOS v7 and later, the SD-WAN configuration syntax changes. Many references to 'virtual-wan-link' become 'sdwan'. See 'Administration Guide: Using BGP tags with SD-WAN rules'.


Solution

 

Diagram:

 



Expectations, and Requirements:

Scenario: HQ has 2 Internet connections called ISP1 and ISP2.

The branch has 1 Internet connection with 2 IPsec tunnels toward HQ ISP1 and HQ ISP2.

  • VPN 1 on ISP1: HQ Tunnel IP 1.1.1.1; Branch Tunnel IP 1.1.1.2.
  • VPN 2 on ISP2: HQ Tunnel IP 2.2.2.1; Branch Tunnel IP 2.2.2.2.

iBGP peering is configured on each VPN.

HQ routing logic :
The branch advertises its subnet with a BGP community:
10.119.0.0/20 → BGP Community 2:1.

HQ applies a route tag depending on the received BGP community:
BGP community 2:1 → route tag 21.

HQ will then apply SD-WAN rules based on the source address and route tag:

  • The route tag 21 and source 10.118.0.0/20 → VPN 1 as the primary path, VPN 2 as the secondary path.
  • The route tag 21 and source 10.111.0.0/20 → VPN 2 as the primary path, VPN 1 as the secondary path.
  • The routes without tag (Internet access) →  port1 as the primary path, port2 as the secondary path.

Branch routing logic :
HQ advertises each subnet with a unique BGP community:
10.118.0.0/20 -> BGP community 1:1.
10.111.0.0/20 -> BGP community 1:2.

The branch applies a route tag depending on the received BGP community:
BGP community 1:1 → route tag 11.
BGP community 1:2 → route tag 12.

The branch will then apply SD-WAN rules based on the route tag:

  • The route tag 11 → VPN 1 as the primary path, VPN 2 as the secondary path.
  • The route tag 12 → VPN 2 as the primary path, VPN 1 as the secondary path
  • The routes without tag (Internet through HQ) → VPN 1 & VPN 2 (default ECMP rule)

SD-WAN path selection is based on the configured SLA.


Configuration.
HQ:

VPNs toward the Branch are already configured as follows:

  • port1_p1 : VPN over ISP1.
  • port2_p1 : VPN over ISP2.

  1. Configure IP addresses on tunnel interfaces for health checks and iBGP peerings:

    config system interface
        edit "port1_p1"
            set ip 1.1.1.1 255.255.255.255
            set remote-ip 1.1.1.2 255.255.255.255
        next
        edit "port2_p1"
            set ip 2.2.2.1 255.255.255.255
            set remote-ip 2.2.2.2 255.255.255.255
        next
    end

  2. port1 (ISP1), port2 (ISP2), port1_p1, port2_p1 are added to SD-WAN.

    Note: port1_p1, and port2_p1 have no gateway since they are tunnel interfaces.

    config system virtual-wan-link
        set status enable
            config members
                edit 1
                    set interface "port1"
                    set gateway 172.31.143.254         
                next
                edit 2
                    set interface "port2"
                    set gateway 172.31.201.13            
                next
                edit 3
                    set interface "port1_p1"
                next
                edit 4
                    set interface "port2_p1"
                next
        end

  3. Configure separate health-checks for Internet connection and IPSec VPNs:

    config system virtual-wan-link
        config health-check
            edit "PingGoogle"
                set server "8.8.8.8"
                set members 1 2
                    config sla
                        edit 1
                        next
                    end
            next
            edit "PingBR"
                set server "10.119.2.36"
                set members 3 4
                    config sla
                        edit 1
                        next
                    end
            next
        end
    end

  4. Configure prefix-lists, route-map, and BGP to set the BGP communities on the advertised routes:

    config router prefix-list
        edit "HQ_LAN_1"
            config rule
                edit 1
                    set prefix 10.118.0.0 255.255.240.0
                    unset ge
                    unset le
                next
            end
        next
        edit "HQ_LAN_2"
            config rule
                edit 1
                    set prefix 10.111.0.0 255.255.240.0
                    unset ge
                    unset le
                next
            end
        next
    end

    config router route-map
        edit "Set_community"
            config rule
                edit 1
                    set match-ip-address "HQ_LAN_1"
                    set set-community "1:1"
                next
                edit 2
                    set match-ip-address "HQ_LAN_2"
                    set set-community "1:2"
                next
                edit 3
                next
            end
        next
    end

    config router bgp
        set as 1
            config neighbor
                edit "1.1.1.2"
                    set remote-as 1
                    set route-map-out "Set_community"
                next
                edit "2.2.2.2"
                    set remote-as 1
                    set route-map-out "Set_community"
                next
            end
            config network
                edit 1
                    set prefix 10.118.0.0 255.255.240.0
                next
                edit 2
                    set prefix 10.111.0.0 255.255.240.0
                next
            end
    end

    Note: route-tag is not a BGP attribute, therefore route-tags for BGP prefixes can only be applied using a route-map-in as is done in step 5. 'set set-route-tag' can be configured in a route-map-out, but doing so will have no effect if applied to a BGP neighbor.

  5. Configure community-lists, route-map, and BGP to set the route-tag depending on the received BGP community:

    config router community-list
        edit "Branch_LAN"
            config rule
                edit 1
                    set action permit
                    set match "2:1"
                next
            end
        next
    end
    config router route-map
        edit "Community_to_tag"
            config rule
                edit 1
                    set match-community "Branch_LAN"
                    set set-route-tag 21
                next
                edit 2
                next
            end
        next
    end

    config router bgp
        config neighbor
            edit "1.1.1.2"
                set route-map-in "Community_to_tag"
            next
            edit "2.2.2.2"
                set route-map-in "Community_to_tag"
            next
        end

  6. Configure SD-WAN rules and address objects:

    config firewall address
        edit "Remote_subnet"
            set subnet 10.119.0.0 255.255.240.0
        next
        edit "Tunnel_port1"
            set subnet 1.1.1.0 255.255.255.252
        next
        edit "Tunnel_port2"
            set subnet 2.2.2.0 255.255.255.252
        next
        edit "HQ_Subnet_1"
            set subnet 10.118.0.0 255.255.240.0
        next
        edit "HQ_Subnet_2"
            set subnet 10.111.0.0 255.255.240.0
        next
    end

    config system virtual-wan-link
        config service
            edit 4
                set name "To_Branch_1"
                set mode sla
                set route-tag 21
                set src "HQ_Subnet_1"
                    config sla
                        edit "PingBR"
                            set id 1
                        next
                    end
                set priority-members 3 4
            next
            edit 3
                set name "To_Branch_2"
                set mode sla
                set route-tag 21
                set src "HQ_Subnet_2"
                    config sla
                        edit "PingBR"
                            set id 1
                        next
                    end
                set priority-members 4 3
            next
            edit 2
                set name "To_Internet"
                set mode sla
                set dst "all"
                    config sla
                        edit "PingGoogle"
                            set id 1
                        next
                    end
                set priority-members 1 2
            next
        end
    end

  7. The default route points to the virtual-wan-link (SD-WAN) interface:

    config router static
        edit 1
            set distance 1
            set virtual-wan-link enable
        next
    end

  8. 4 Firewall policies are created to allow the traffic as follows:

  • From LAN HQ to Branch (ID 5).
  • From Branch to LAN HQ 1 (ID 4).
  • From Branch to LAN HQ 2 (ID 7).
  • From LAN HQ to Internet (ID 3).
  • From Branch to Internet (ID 6).

 

config firewall policy
    edit 5
        set name "LAN_HQ->BR"
        set srcintf "port5" "port6"
        set dstintf "virtual-wan-link"
        set srcaddr "all"
        set dstaddr "Remote_subnet"
        set action accept
        set schedule "always"
        set service "ALL"
    next
    edit 4
        set name "BR->LAN_HQ"
        set srcintf "virtual-wan-link"
        set dstintf "port5"
        set srcaddr "Remote_subnet" "Tunnel_port1" "Tunnel_port2"
        set dstaddr "HQ_Subnet_1"
        set action accept
        set schedule "always"
        set service "ALL"
    next
    edit 7
        set name "BR->LAN_HQ_2"
        set srcintf "virtual-wan-link"
        set dstintf "port6"
        set srcaddr "Remote_subnet"
        set dstaddr "HQ_Subnet_2"
        set action accept
        set schedule "always"
        set service "ALL"
    next
    edit 3
        set name "LAN_HQ->Internet"
        set srcintf "port5" "port6"
        set dstintf "virtual-wan-link"
        set srcaddr "all"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
        set utm-status enable
        set logtraffic all
        set av-profile "default"
        set webfilter-profile "monitor-all"
        set ips-sensor "default"
        set application-list "default"
        set profile-protocol-options "default"
        set ssl-ssh-profile "certificate-inspection"
        set nat enable
    next
    edit 6
        set name "BR->Internet"
        set srcintf "virtual-wan-link"
        set dstintf "virtual-wan-link"
        set srcaddr "Remote_subnet"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
        set utm-status enable
        set av-profile "default"
        set webfilter-profile "monitor-all"
        set ips-sensor "default"
        set application-list "default"
        set profile-protocol-options "default"
        set ssl-ssh-profile "certificate-inspection"
        set nat enable
    next
end

 

Branch:

VPNs toward HQ are already configured as follows:
to_port1_p1: VPN toward HQ ISP1.
to_port2_p1: VPN toward HQ ISP2.

 

  1. Configure IP addresses on tunnel interfaces for health checks:

    config system interface
        edit "to_port1_p1"
            set ip 1.1.1.2 255.255.255.255
            set remote-ip 1.1.1.1 255.255.255.255
        next
        edit "to_port2_p1"
            set ip 2.2.2.2 255.255.255.255
            set remote-ip 2.2.2.1 255.255.255.255
        next
    end

  2. to_port1_p1, to_port2_p1 are added to SD-WAN:

    config system virtual-wan-link
        set status enable
            config members
                edit 1
                    set interface "to_port1_p1"
                next
                edit 2
                    set interface "to_port2_p1"
                next
        end

  3. Configure health-checks for IPSec VPNs:

    config system virtual-wan-link
        config health-check
            edit "PingHQFGT"
                set server "10.118.3.5"
                set members 1 2
                    config sla
                        edit 1
                            set packetloss-threshold 1
                        next
                    end
            next
        end
    end

  4. Configure prefix-lists, route-map, and BGP to set the BGP communities on the advertised routes:

    config router prefix-list
        edit "Branch_LAN"
            config rule
                edit 1
                    set prefix 10.119.0.0 255.255.240.0
                    unset ge
                    unset le
                next
            end
        next
    end

    config router route-map
        edit "Set_community"
            config rule
                edit 1
                    set match-ip-address "Branch_LAN"
                    set set-community "2:1"
                next
                edit 2
                next
            end
        next
    end

    config router bgp
        set as 1
            config neighbor
                edit "1.1.1.1"
                    set remote-as 1
                    set route-map-out "Set_community"
                next
                edit "2.2.2.1"
                    set remote-as 1
                    set route-map-out "Set_community"
                next
            end
            config network
                edit 1
                    set prefix 10.119.0.0 255.255.240.0
                next
    end

    Note: route-tag is not a BGP attribute, therefore route-tags for BGP prefixes can only be applied using a route-map-in as is done in step 5. 'set set-route-tag' can be configured in a route-map-out, but doing so will have no effect if applied to a BGP neighbor.

  5. Configure community-lists, route-map, and BGP to set the route-tag depending on the received BGP community:

    config router community-list
        edit "HQ_LAN_1"
            config rule
                edit 1
                    set action permit
                    set match "1:1"
                next
            end
        next
        edit "HQ_LAN_2"
            config rule
                edit 1
                    set action permit
                    set match "1:2"
                next
            end
        next
    end

    config router route-map
        edit "Community_to_tag"
            config rule
                edit 1
                    set match-community "HQ_LAN_1"
                    set set-route-tag 11
                next
                edit 2
                    set match-community "HQ_LAN_2"
                    set set-route-tag 12
                next
                edit 3
                next
            end
        next
    end

    config router bgp
        config neighbor
            edit "1.1.1.1"
                set route-map-in "Community_to_tag"
            next
            edit "2.2.2.1"
                set route-map-in "Community_to_tag"
            next
        end

  6. Configure SD-WAN rules:

    config system virtual-wan-link
        config service
            edit 2
                set name "To_HQ_2"
                set mode sla
                set route-tag 12
                    config sla
                        edit "PingHQFGT"
                            set id 1
                        next
                    end
                set priority-members 2 1
            next
            edit 1
                set name "To_HQ_1"
                set mode sla
                set route-tag 11
                    config sla
                        edit "PingHQFGT"
                            set id 1
                        next
                    end
                set priority-members 1 2
            next
        end
    end

  7. The default route points to the virtual-wan-link (SD-WAN) interface. Routes toward the remote VPN gateway are added on wan1 to establish the VPN tunnels:

    config router static
        edit 2
            set dst 172.31.195.5 255.255.255.255
            set gateway 10.5.31.254
            set device "wan1"
        next
        edit 3
            set dst 172.31.131.5 255.255.255.255
            set gateway 10.5.31.254
            set device "wan1"
        next
        edit 4
            set distance 1
            set virtual-wan-link enable
        next
    end

  8. 2 Firewall policies are created on the SD-WAN interface to allow the traffic as follows:
  • From Branch to 'any' (ID 1).
  • From 'any internal IPs' to branch (ID 2).

 

config firewall address
    edit "HQ_Subnet_1"
        set subnet 10.118.0.0 255.255.240.0
    next
    edit "Tunnel_1"
        set subnet 1.1.1.0 255.255.255.252
    next
    edit "Tunnel_2"
        set subnet 2.2.2.0 255.255.255.252
    next
    edit "HQ_Subnet_2"
        set subnet 10.111.0.0 255.255.240.0
    next
end

config firewall policy
    edit 1
        set name "BR->HQ"
        set srcintf "internal"
        set dstintf "virtual-wan-link"
        set srcaddr "all"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
    next
    edit 2
        set name "HQ->BR"
        set srcintf "virtual-wan-link"
        set dstintf "internal"
        set srcaddr "HQ_Subnet_1" "Tunnel_1" "Tunnel_2" "HQ_Subnet_2"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
    next
end


Verification.
HQ:

Received BGP networks are properly tagged:

 

FG300E-HQ # get router info bgp network 10.119.0.0
BGP routing table entry for 10.119.0.0/20
Paths: (2 available, best #1, table Default-IP-Routing-Table)
  Not advertised to any peer
  Local
    1.1.1.2 from 1.1.1.2 (10.119.2.36)
      Origin IGP metric 0, localpref 100, valid, internal, best
      Community: 2:1
      Last update: Mon Apr 16 09:50:35 2018

  Local
    2.2.2.2 from 2.2.2.2 (10.119.2.36)
      Origin IGP metric 0, localpref 100, valid, internal
      Community: 2:1
      Last update: Mon Apr 16 09:50:39 2018


FG300E-HQ # get router info bgp network
BGP table version is 22, local router ID is 10.10.10.10
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight RouteTag Path
*> 10.111.0.0/20    0.0.0.0                       100  32768        0 i
*> 10.118.0.0/20    0.0.0.0                       100  32768        0 i
*>i10.119.0.0/20    1.1.1.2                  0    100      0       21 i
* i                 2.2.2.2                  0    100      0       21 i

Total number of prefixes 3

 

SD-WAN rules path selection:

 

FG300E-HQ # diag sys virtual-wan-link service

Service(4): Address Mode(IPV4) flags=0x0
  TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla)
  Members:
    1: Seq_num(3), alive, sla(0x1),cfg_order(0), selected
    2: Seq_num(4), alive, sla(0x1),cfg_order(1), selected
  Src address: 10.118.0.0-10.118.15.255
  Route tag address: 10.119.0.0/255.255.240.0

Service(3): Address Mode(IPV4) flags=0x0
  TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla)
  Members:
    1: Seq_num(4), alive, sla(0x1),cfg_order(0), selected
    2: Seq_num(3), alive, sla(0x1),cfg_order(1), selected
  Src address: 10.111.0.0-10.111.15.255
  Route tag address: 10.119.0.0/255.255.240.0

Service(2): Address Mode(IPV4) flags=0x0
  TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla)
  Members:
    1: Seq_num(1), alive, sla(0x1),cfg_order(0), selected
    2: Seq_num(2), alive, sla(0x1),cfg_order(1), selected
  Dst address: 0.0.0.0-255.255.255.255

 

FG300E-HQ # diagnose firewall proute list
list route policy info(vf=root):

id=4280680452 vwl_service=4(To_Branch_1) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0:65535 iif=0 dport=1-65535 oif=44 oif=45
source(1): 10.118.0.0-10.118.15.255
destination wildcard(1): 10.119.0.0/255.255.240.0

id=4280680451 vwl_service=3(To_Branch_2) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0:65535 iif=0 dport=1-65535 oif=45 oif=44
source(1): 10.111.0.0-10.111.15.255
destination wildcard(1): 10.119.0.0/255.255.240.0


id=4280680450 vwl_service=2(To_Internet) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0:65535 iif=0 dport=1-65535 oif=9 oif=10
destination(1): 0.0.0.0-255.255.255.255
source wildcard(1): 0.0.0.0/0.0.0.0

 

Branch:

Received BGP networks are properly tagged:

 

FG60D-BR # get router info bgp network 10.118.0.0
BGP routing table entry for 10.118.0.0/20

Paths: (2 available, best #1, table Default-IP-Routing-Table)
  Not advertised to any peer
  Local
    1.1.1.1 from 1.1.1.1 (10.10.10.10)
      Origin IGP metric 0, localpref 100, valid, internal, best
      Community: 1:1
      Last update: Mon Apr 16 09:21:17 2018

  Local
    2.2.2.1 from 2.2.2.1 (10.10.10.10)
      Origin IGP metric 0, localpref 100, valid, internal
      Community: 1:1
      Last update: Mon Apr 16 09:21:31 2018

FG60D-BR # get router info bgp network 10.111.0.0
BGP routing table entry for 10.111.0.0/20
Paths: (2 available, best #1, table Default-IP-Routing-Table)
  Not advertised to any peer
  Local
    1.1.1.1 from 1.1.1.1 (10.10.10.10)
      Origin IGP metric 0, localpref 100, valid, internal, best
      Community: 1:2
      Last update: Mon Apr 16 09:21:17 2018

  Local
    2.2.2.1 from 2.2.2.1 (10.10.10.10)
      Origin IGP metric 0, localpref 100, valid, internal
      Community: 1:2
      Last update: Mon Apr 16 09:21:31 2018

FG60D-BR # get router info bgp network
BGP table version is 8, local router ID is 10.119.2.36
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight RouteTag Path
*>i10.111.0.0/20    1.1.1.1                  0    100      0       12 i
* i                 2.2.2.1                  0    100      0       12 i
*>i10.118.0.0/20    1.1.1.1                  0    100      0       11 i
* i                 2.2.2.1                  0    100      0       11 i
*> 10.119.0.0/20    0.0.0.0                       100  32768        0 i

FG60D-BR # diagnose sys virtual-wan-link service
Service(2): Address Mode(IPV4) flags=0x0
  TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla)
  Members:
    1: Seq_num(2), alive, sla(0x1),cfg_order(0), selected
    2: Seq_num(1), alive, sla(0x1),cfg_order(1), selected
  Route tag address: 10.111.0.0/255.255.240.0

Service(1): Address Mode(IPV4) flags=0x0
  TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla)
  Members:
    1: Seq_num(1), alive, sla(0x1),cfg_order(0), selected
    2: Seq_num(2), alive, sla(0x1),cfg_order(1), selected
  Route tag address: 10.118.0.0/255.255.240.0


FG60D-BR # diagnose firewall proute list
list route policy info(vf=root):

id=4279697410 vwl_service=2(To_HQ_2) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0:65535 iif=0 dport=1-65535 oif=20 oif=19
source wildcard(1): 0.0.0.0/0.0.0.0
destination wildcard(1): 10.111.0.0/255.255.240.0

id=4279697409 vwl_service=1(To_HQ_1) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0:65535 iif=0 dport=1-65535 oif=19 oif=20
source wildcard(1): 0.0.0.0/0.0.0.0
destination wildcard(1): 10.118.0.0/255.255.240.0

 

Related article:

Technical Tip: Configure FortiGate SD-WAN with an IPSEC VPN